Adaptive Sparse Gaussian Process.

Adaptive learning is necessary for nonstationary environments where the learning machine needs to forget past data distribution. Efficient algorithms require a compact model update to not grow in computational burden with the incoming data and with the lowest possible computational cost for online parameter updating. Existing solutions only partially cover these needs. Here, we propose the first adaptive sparse Gaussian process (GP) able to address all these issues. We first reformulate a variational sparse GP (VSGP) algorithm to make it adaptive through a forgetting factor. Next, to make the model inference as simple as possible, we propose updating a single inducing point of the SGP model together with the remaining model parameters every time a new sample arrives. As a result, the algorithm presents a fast convergence of the inference process, which allows an efficient model update (with a single inference iteration) even in highly nonstationary environments. Experimental results demonstrate the capabilities of the proposed algorithm and its good performance in modeling the predictive posterior in mean and confidence interval estimation compared to state-of-the-art approaches.

Multidimensional analysis of immune cells from COVID-19 patients identified cell subsets associated with the severity at hospital admission.

BACKGROUND: SARS-CoV-2 emerged as a new coronavirus causing COVID-19, and it has been responsible for more than 760 million cases and 6.8 million deaths worldwide until March 2023. Although infected individuals could be asymptomatic, other patients presented heterogeneity and a wide range of symptoms. Therefore, identifying those infected individuals and being able to classify them according to their expected severity could help target health efforts more effectively. METHODOLOGY/PRINCIPAL FINDINGS: Therefore, we wanted to develop a machine learning model to predict those who will develop severe disease at the moment of hospital admission. We recruited 75 individuals and analysed innate and adaptive immune system subsets by flow cytometry. Also, we collected clinical and biochemical information. The objective of the study was to leverage machine learning techniques to identify clinical features associated with disease severity progression. Additionally, the study sought to elucidate the specific cellular subsets involved in the disease following the onset of symptoms. Among the several machine learning models tested, we found that the Elastic Net model was the better to predict the severity score according to a modified WHO classification. This model was able to predict the severity score of 72 out of 75 individuals. Besides, all the machine learning models revealed that CD38+ Treg and CD16+ CD56neg HLA-DR+ NK cells were highly correlated with the severity. CONCLUSIONS/SIGNIFICANCE: The Elastic Net model could stratify the uninfected individuals and the COVID-19 patients from asymptomatic to severe COVID-19 patients. On the other hand, these cellular subsets presented here could help to understand better the induction and progression of the symptoms in COVID-19 individuals.

Multi-task longitudinal forecasting with missing values on Alzheimer's disease.

BACKGROUND AND OBJECTIVE: Machine learning techniques typically used in dementia assessment are not able to learn multiple tasks jointly and deal with time-dependent heterogeneous data containing missing values. In this paper, we reformulate SSHIBA, a recently introduced Bayesian multi-view latent variable model, for jointly learning diagnosis, ventricle volume, and ADAS score in dementia on longitudinal data with missing values. METHODS: We propose a novel Bayesian Variational inference framework capable of simultaneously imputing missing values and combining information from several views. This way, we can combine different data views from different time-points in a common latent space and learn the relationships between each time-point, using the semi-supervised formulation to fully exploit the temporal structure of the data and handle missing values. In turn, the model can combine all the available information to simultaneously model and predict multiple output variables. RESULTS: We applied the proposed model to jointly predict diagnosis, ventricle volume, and ADAS score in dementia. The comparison of imputation strategies demonstrated the superior performance of the semi-supervised formulation of the model, improving the best baseline methods. Moreover, the performance in simultaneous prediction of diagnosis, ventricle volume, and ADAS score led to an improved prediction performance over the best baseline method. CONCLUSIONS: The results demonstrate that the proposed SSHIBA framework can learn an excellent imputation of the missing values and outperforming the baselines while simultaneously predicting three different tasks.

Regularized Bagged Canonical Component Analysis for Multiclass Learning in Brain Imaging.

A fundamental problem of supervised learning algorithms for brain imaging applications is that the number of features far exceeds the number of subjects. In this paper, we propose a combined feature selection and extraction approach for multiclass problems. This method starts with a bagging procedure which calculates the sign consistency of the multivariate analysis (MVA) projection matrix feature-wise to determine the relevance of each feature. This relevance measure provides a parsimonious matrix, which is combined with a hypothesis test to automatically determine the number of selected features. Then, a novel MVA regularized with the sign and magnitude consistency of the features is used to generate a reduced set of summary components providing a compact data description. We evaluated the proposed method with two multiclass brain imaging problems: 1) the classification of the elderly subjects in four classes (cognitively normal, stable mild cognitive impairment (MCI), MCI converting to AD in 3 years, and Alzheimer's disease) based on structural brain imaging data from the ADNI cohort; 2) the classification of children in 3 classes (typically developing, and 2 types of Attention Deficit/Hyperactivity Disorder (ADHD)) based on functional connectivity. Experimental results confirmed that each brain image (defined by 29.852 features in the ADNI database and 61.425 in the ADHD) could be represented with only 30 - 45% of the original features. Furthermore, this information could be redefined into two or three summary components, providing not only a gain of interpretability but also classification rate improvements when compared to state-of-art reference methods.

Sign-Consistency Based Variable Importance for Machine Learning in Brain Imaging.

An important problem that hinders the use of supervised classification algorithms for brain imaging is that the number of variables per single subject far exceeds the number of training subjects available. Deriving multivariate measures of variable importance becomes a challenge in such scenarios. This paper proposes a new measure of variable importance termed sign-consistency bagging (SCB). The SCB captures variable importance by analyzing the sign consistency of the corresponding weights in an ensemble of linear support vector machine (SVM) classifiers. Further, the SCB variable importances are enhanced by means of transductive conformal analysis. This extra step is important when the data can be assumed to be heterogeneous. Finally, the proposal of these SCB variable importance measures is completed with the derivation of a parametric hypothesis test of variable importance. The new importance measures were compared with a t-test based univariate and an SVM-based multivariate variable importances using anatomical and functional magnetic resonance imaging data. The obtained results demonstrated that the new SCB based importance measures were superior to the compared methods in terms of reproducibility and classification accuracy.

Comparison of feature representations in MRI-based MCI-to-AD conversion prediction.

Alzheimer's disease (AD) is a progressive neurological disorder in which the death of brain cells causes memory loss and cognitive decline. The identification of at-risk subjects yet showing no dementia symptoms but who will later convert to AD can be crucial for the effective treatment of AD. For this, Magnetic Resonance Imaging (MRI) is expected to play a crucial role. During recent years, several Machine Learning (ML) approaches to AD-conversion prediction have been proposed using different types of MRI features. However, few studies comparing these different feature representations exist, and the existing ones do not allow to make definite conclusions. We evaluated the performance of various types of MRI features for the conversion prediction: voxel-based features extracted based on voxel-based morphometry, hippocampus volumes, volumes of the entorhinal cortex, and a set of regional volumetric, surface area, and cortical thickness measures across the brain. Regional features consistently yielded the best performance over two classifiers (Support Vector Machines and Regularized Logistic Regression), and two datasets studied. However, the performance difference to other features was not statistically significant. There was a consistent trend of age correction improving the classification performance, but the improvement reached statistical significance only rarely.

Discovering brain regions relevant to obsessive-compulsive disorder identification through bagging and transduction.

In the present study we applied a multivariate feature selection method based on the analysis of the sign consistency of voxel weights across bagged linear Support Vector Machines (SVMs) with the aim of detecting brain regions relevant for the discrimination of subjects with obsessive-compulsive disorder (OCD, n=86) from healthy controls (n=86). Each participant underwent a structural magnetic resonance imaging (sMRI) examination that was pre-processed in Statistical Parametric Mapping (SPM8) using the standard pipeline of voxel-based morphometry (VBM) studies. Subsequently, we applied our multivariate feature selection algorithm, which also included an L2 norm regularization to account for the clustering nature of MRI data, and a transduction-based refinement to further control overfitting. Our approach proved to be superior to two state-of-the-art feature selection methods (i.e., mass-univariate t-Test selection and recursive feature elimination), since, following the application of transductive refinement, we obtained a lower test error rate of the final classifier. Importantly, the regions identified by our method have been previously reported to be altered in OCD patients in studies using traditional brain morphometry methods. By contrast, the discrimination patterns obtained with the t-Test and the recursive feature elimination approaches extended across fewer brain regions and included fewer voxels per cluster. These findings suggest that the feature selection method presented here provides a more comprehensive characterization of the disorder, thus yielding not only a superior identification of OCD patients on the basis of their brain anatomy, but also a discrimination map that incorporates most of the alterations previously described to be associated with the disorder.

A multiple kernel learning approach to perform classification of groups from complex-valued fMRI data analysis: application to schizophrenia.

FMRI data are acquired as complex-valued spatiotemporal images. Despite the fact that several studies have identified the presence of novel information in the phase images, they are usually discarded due to their noisy nature. Several approaches have been devised to incorporate magnitude and phase data, but none of them has performed between-group inference or classification. Multiple kernel learning (MKL) is a powerful field of machine learning that finds an automatic combination of kernel functions that can be applied to multiple data sources. By analyzing this combination of kernels, the most informative data sources can be found, hence providing a better understanding of the analyzed learning task. This paper presents a methodology based on a new MKL algorithm (ν-MKL) capable of achieving a tunable sparse selection of features' sets (brain regions' patterns) that improves the classification accuracy rate of healthy controls and schizophrenia patients by 5% when phase data is included. In addition, the proposed method achieves accuracy rates that are equivalent to those obtained by the state of the art lp-norm MKL algorithm on the schizophrenia dataset and we argue that it better identifies the brain regions that show discriminative activation between groups. This claim is supported by the more accurate detection achieved by ν-MKL of the degree of information present on regions of spatial maps extracted from a simulated fMRI dataset. In summary, we present an MKL-based methodology that improves schizophrenia characterization by using both magnitude and phase fMRI data and is also capable of detecting the brain regions that convey most of the discriminative information between patients and controls.

Analysis of fMRI time series with mutual information.

Neuroimaging plays a fundamental role in the study of human cognitive neuroscience. Functional magnetic resonance imaging (fMRI), based on the Blood Oxygenation Level Dependent signal, is currently considered as a standard technique for a system level understanding of the human brain. The problem of identifying regionally specific effects in neuroimaging data is usually solved by applying Statistical Parametric Mapping (SPM). Here, a mutual information (MI) criterion is used to identify regionally specific effects produced by a task. In particular, two MI estimators are presented for its use in fMRI data. The first one uses a Parzen probability density estimation, and the second one is based on a K Nearest Neighbours (KNN) estimation. Additionally, a statistical measure has been introduced to automatically detect the voxels which are relevant to the fMRI task. Experiments demonstrate the advantages of MI estimators over SPM maps; firstly, providing more significant differences between relevant and irrelevant voxels; secondly, presenting more focalized activation; and, thirdly, detecting small areas related to the task. These findings, and the improved performance of KNN MI estimator in multisubject and multistimuli studies, make the proposed methods a good alternative to SPM.

Real AdaBoost with gate controlled fusion.

In this brief, we propose to increase the capabilities of standard real AdaBoost (RAB) architectures by replacing their linear combinations with a fusion controlled by a gate with fixed kernels. Experimental results in a series of well-known benchmark problems support the effectiveness of this approach in improving classification performance. Although the need for cross-validation processes obviously leads to higher training requirements and more computational effort, the operation load is never much higher; in many cases it is even lower than that of competitive RAB schemes.

Support vector machines with constraints for sparsity in the primal parameters.

This paper introduces a new support vector machine (SVM) formulation to obtain sparse solutions in the primal SVM parameters, providing a new method for feature selection based on SVMs. This new approach includes additional constraints to the classical ones that drop the weights associated to those features that are likely to be irrelevant. A ν-SVM formulation has been used, where ν indicates the fraction of features to be considered. This paper presents two versions of the proposed sparse classifier, a 2-norm SVM and a 1-norm SVM, the latter having a reduced computational burden with respect to the first one. Additionally, an explanation is provided about how the presented approach can be readily extended to multiclass classification or to problems where groups of features, rather than isolated features, need to be selected. The algorithms have been tested in a variety of synthetic and real data sets and they have been compared against other state of the art SVM-based linear feature selection methods, such as 1-norm SVM and doubly regularized SVM. The results show the good feature selection ability of the approaches.

A dynamically adjusted mixed emphasis method for building boosting ensembles.

Progressively emphasizing samples that are difficult to classify correctly is the base for the recognized high performance of real Adaboost (RA) ensembles. The corresponding emphasis function can be written as a product of a factor that measures the quadratic error and a factor related to the proximity to the classification border; this fact opens the door to explore the potential advantages provided by using adjustable combined forms of these factors. In this paper, we introduce a principled procedure to select the combination parameter each time a new learner is added to the ensemble, just by maximizing the associated edge parameter, calling the resulting method the dynamically adapted weighted emphasis RA (DW-RA). A number of application examples illustrates the performance improvements obtained by DW-RA.